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1 Introduction 

The problem of extremum seeking navigation is concerned with driving a mobile robot to the maximizer of an 
environmental field. The field profile is unknown a priori and should be explored on-line based on point-wise 
measurements of the field. This scenario arises in a variety of missions, such as environmental studies, tracking 
a moving target based on a single signal decaying away from the target, seeking sources of radiation emission 
or pollutant leakage, etc. The discussed problem falls into the general framework of extremum seeking control, 
which should ensure that a dynamic plant is directed to and subsequently tracks an optimal operating point 
based on on-line evaluation of the current performance, whose profile is unknown a priori. 

For recent surveys on extremum seeking control methods and algorithms, we refer the reader to [12II19II31] . A 
good deal of related research was concerned with gradient climbing based on direct on-line gradient estimation 
[U|T1[1T1|35J[3^. This approach is especially beneficial for mobile sensor networks thanks to collaborative field 
measurements in many locations and data exchange [siEiiiiiiiaiizHsn]- However even in this scenario, data 
exchange degradation due to e.g., communication constraints may require each robot in the team to operate 
autonomously over considerable time. Similar algorithms can be basically used for a single robot equipped with 
several sensors that are distant enough from each other and thus provide the field values at several essentially 
diverse locations. In any case, multiple vehicle/sensor scenario means complicated and costly hardware. This 
paper is focused on another case, where at any time, the robot has access to only a single field measurement at 
its current location. 

The lack of multiple sensor data can be compensated via exploring multiple nearby locations by ” dithering” 
the position of the single sensor during special maneuvers, which may be excited by probing high-frequency 
sinusoidal [71191101134] or stochastic [22] inputs. An approach that is similar in spirit is extremum seeking by means 
of many robots performing biased random walks [26] or by two robots with access to relative positions of each 
other and rotational actuation m- These methods rely, either implicitly or explicitly, on systematic sideways 
exploration maneuvers to collect rich enough data. Another approach limits the field gradient information to 
only the time-derivative of the measured field value obtained via e.g., numerical differentiation, [^[5l l23l[2^ and 
partly employs switching controllers dMi]. These give rise to concerns about amplification of the measurement 
noises and chattering, respectively. Though the respective potential detrimental consequences can be successfully 
avoided in the scenarios examined in these papers, anyhow the need to ensure this puts extra burden on controller 
parameters tuning. Adaptive extremum seeking approach m assumes that the field is known up to finitely 
many uncertain steady parameters, which may exceed the real level of knowledge in some applications. 

The previous extremum seeking research dealt mainly with steady fields. However in the real world, envi¬ 
ronmental fields are almost never steady and often cannot be well approximated by steady fields. The problem 
of extremum seeking in dynamic fields is also concerned with navigation and guidance of a mobile robot towards 
an unknowingly maneuvering target based on a single measurement that decays as the sensor goes away from 
the target, like the strength of the infrared, acoustic, or electromagnetic signal, or minus the distance to the 
target. Such navigation is of interest in many areas [Tl|T4l|24]; it carries a potential to reduce the hardware 
complexity and cost and improve target pursuit reliability. A solution for such problem in the very special case 
of the unsteady field — minus the distance to an unknownly moving Dubins-like target — was proposed and 
justified in m- However the results of m are not applicable to more general dynamic fields. 

Unlike the previous research, this paper deals with generic dynamic fields. In this context, it justifies a new 
kinematic control paradigm that offers to keep the velocity orientation angle proportional to the discrepancy 
between the field value and a given linear ascending function of time, as opposed to conventionally trying to 
align the velocity vector with the gradient. This control law is free from evaluation of any field-derivative data, 
uses only finite gains instead of switching control, and demands only minor memory and computational robot’s 
capacities, being reactive in its nature. We provide a mathematically rigorous evidence of convergence of this 
control law in the case of a generic dynamic field. In doing so, its is shown that the closed-loop system is prone 
to monotonic, non-oscillatory behavior during the transient to the field maximizer provided that the controller 
parameters are properly tuned. We offer recommendations on the choice of these parameters under which the 
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robot inevitably reaches the desired vicinity of the moving field maximizer in a finite time and remains there 
afterwards. 

An algorithm similar to ours can be found in which however uses the estimated time-derivative of the 
measurement. Another difference is that in [2, only a steady harmonic field was examined, the performance 
during the transient and the behavior after reaching a vicinity of the maximizer were not addressed even for 
general harmonic fields, and the convergence conditions were partly implicit by giving no explicit bound on 
some entities that were assumed sufficiently large. The focus of this paper is on general dynamic fields, and we 
offer a study of the entire maneuver with explicit conditions for convergence. 

The body of the paper is organized as follows. Section[2]presents the problem setup and control law. Section[3] 
discloses the closed-loop behavior in a simple but instructive case of a linear field. The main theoretical results 
are given in Section 31 where a generic dynamic field is considered. These results are illustrated by examples in 
Section [5l and are confirmed and supplemented via simulation tests and experiments with a real wheeled robot 
in Sections ?? and ??, respectively. Section ?? offers brief conclusions. The proofs of all theoretical results are 
given in three appendices. 

The extended introduction and discussion of the proposed control law are given in the paper submitted by 
the authors to the IEEE Transactions on Control Systems Technology. This text basically contains the proofs 
of the technical facts underlying justification of the convergence at performance of the proposed algorithm in 
that paper, which were not included into it due to the length limitations. To make the current text logically 
consistent, we reproduce the problem statement and notations. 


2 Problem Setup and the Control Algorithm 

We consider a point-wise robot traveling in a Cartesian plane with the absolute coordinates x and y. The robot 
is controlled by the time-varying linear velocity v whose magnitude does not exceed a given constant v. The 
plane hosts an unknown scalar time-varying field D{t, r) € R, where t is time and r := (x, y)^ ■ The objective is 
to drive the robot to the point r^(t) where D(t, r) attains its maximum over r and then to keep it in a vicinity 
of r°(f), thus displaying the approximate location of r°{t). The on-board control system has access only to the 
field value d{t) := D[t,r{t)] at the robot current location r{t) = [x{t),y{t)]^. No data about the derivatives of 
D are available; in particular, the robot is aware of neither the gradient of D{-) nor the time-derivative d of the 
measurement d. 

The kinematic model of the robot is as follows: 

r =v, r(0) = Tin ||?f|| < V, (2.1) 

where || • || is the Euclidian norm. The problem is to design a controller that drives the robot into the desired 
vicinity T*(i) of the time-varying maximizer r^(t) in a finite time to and then keeps the robot within 
In this paper, we examine the following control algorithm: 

v{t) =ve^fj,[d{t) — do{t)]'^, where do(t) := i/t-f d* and e(0):=(^?^^) (2.2) 

is the unit vector with the heading angle 9 and > 0,/i > 0, d, S R are parameters of the controller. The 
control law (12.21) keeps the robot heading proportional to the discrepancy d{t) — do{t) between the current field 
value d{t) and the linear reference signal do(-) that ascends at the rate of ly and is interpreted as a desired or 
requested behavior of the Held over the robot’s trajectory. 


3 Sample Behavior in a Linear Field 


We start our study of the proposed control law from analysis of its effect in steady linear fields 


D{x,y) = n(^xcosip + ysiiKfi) + Do, where n > 0,and Hq G R (3.1) 

are given. This is instructive since any smooth dynamic field is well approximated by a linear steady field in a 
sufficiently small (and sometimes not so small) region of space-time. So such analysis discloses basic behavioral 
primitives that underlay, more or less, the closed-loop performance in generic fields. We first assume that the 
requested field ascending rate iz = do is feasible. For vehicles traveling at speeds no greater than v, feasible 
rates d do not exceed vn. So we first assume that zz < vn; the opposite case will be addressed in Lemma l3. 11 
We first note that perfect tracking d{t) = do{t) of the reference signal is not urgent: for gradient climbing, 
it suffices to ensure ascend of the field value d at the requested rate zz. Since 


d = n^x cos ip + y simpj = nx(cos d cos -I-sm d sm j nv cos{9 — p), 


(3.2) 
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this is possible only if the robot moves over a straight line subtending an angle of = ip± arccos ^ + 27rfc with 
the x-axis, where k is an integer. In turn, such a motion conforms to the control law (12.211 if and only if 

= d{t) — do{t) = n [(xin + tvcosO^) cosip + (yin + tI7sin0|) sin(/?] — vt — d^, 

= [nvcos{ip — 9-^) — v] -t + D{rin) — d* = D{rin) — d* 

=0 

or, in other words, 

y, [Zl(rin) — d*] =9* := (^ ± 0* + 27rfc, 0* := arccos —. (3.3) 

vn 

Thus any line L subtending an angle of 0* with the field gradient accommodates inhnitely many “perfect” 
trajectories of the closed-loop system; their characteristic feature is that the requested field growth rate v is 
tracked perfectly d=v. These processes are enumerated by fc = 0, ±1,...; the fcth of them is at the speed v 
along the line L and from the unique point Tin £ L that solves the linear equation (13.31) : see Figs. [TKa,b). The 
starting points related to various k are equally spaced and separated by the distance 2TY/(yn). 




Figure 1: (a,b) “Perfect” closed-loop trajectories (a) 0 = (/? -I- 0* and (b) 0 = — 0*; (c) Convergence to a 

“perfect” trajectory. 


The following theorem summarizes the above analysis and addresses stability of “perfect” processes. It also 
shows that the properly tuned controller makes the system pre-disposed to monotonic, unfluctuating behavior. 

Theorem 3.1. Let the system (EB be driven by the controller (12.211 in the steady linear field dSU, and the 
requested field growth rate be feasible v < vn. Depending on the initial state Tin, all trajectories are divided into 
two categories: 

i) Let Tin be such that (liOll holds. Then the robot moves at the speed v along a straight line subtending the 
angle 0* from (lT3ll with the field gradient. The field value d constantly grows at the requested rate: d = i/; 

ii) If (13.311 does not hold, the trajectory of the robot, including the velocity orientation, exponentially converges 
to a “perfect” trajectory described in i) and Fig. UKa). Meanwhile, the field value d eventually ascends and 
ultimately approaches the requested ascending rate d(t) —>■ z/ ast^oo. As t runs from 0 to oo, the velocity 
orientation 9 monotonically sweeps through an angle that is less than 2 tt, like in Fig. [IJc). 

Proof: With a proper rotation of the coordinate frame in mind, it can be assumed that (/? = 0 in (13.111 . 

i) The claim summarizes the study of “perfect” trajectories carried out in the foregoing. 

ii) We start with observing that 


0 y[d — do\ = y[nvcos9—i/\ = yvnx{9), 


where x(0) := cos0 — cos0*. 


(3.4) 


Here x(-) has inhnitely many roots 9±(k) = ±0* -I- 27rfc, fc = 0, ±1,... (see Fig.j^Ka)) and ±x^[0±(fc)] < 0. Hence 
0+(fc) and 0-(fc) are locally stable and unstable equilibria, respectively, and any solution 0(-) ^ &-(k) Vfc of 
(IMJ exponentially and monotonically converges to 0+(fc) with some integer k, moving within an interval lying 
within two adjacent equilibria m- It remains to note that this convergence 9{t) —>■ 0+(fc) implies the exponential 
convergence r(f) — r+(f) —>■ 0 of the trajectories r(t) = v Jq ^^(s)] ds + Vin and r+(t) = v Jq e[9+(k)] ds + r*, 
where r* := Tin -l-U/g°° {e[0(s)] — e[0+(fc)]} ds satishes (13.311 (with Tin := = O,0'|- := 0+(fc)): 


nOO 

fi[D{r^) — = fj.[D{rin) — d^] fivn / [cos^( 5 ) — cos(A:)] ds 

Jo 

nCO 

- fJ.[d(0) — d*] -I- fivn / [cos0(s) — cos0*] ds 

Jo 


101. 


In t—¥(yD 


□ 
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Claim ii) implies that any “perfect” trajectory described in i) and Fig. [TJb) is unstable, contrary to those 
from Fig. [ija). By the last sentence from ii), the robot does not oscillate and does not make full turns since 
oscillation violates monotonic evolution of 0, whereas a full turn means that 9 runs the length of 2tt. This 
monotonic pattern is antipodal to perpetual oscillations caused by the controllers from [TnunmiiiiiiaiM] 
even in linear fields. 

By retracing the arguments from the proof of Theorem 13.11 it is easy to see that in the marginal case of the 
maximal feasible field-rate request v = Fn, the robot still successfully achieves this rate d(t) —>■ z/ as t —>■ oo, 
with retaining the monotonic behavior described by the last sentence from ii). 

As for the case where unrealistic field growth rate is requested v > vn, the following lemma shows that even 
in this case, motion in the gradient direction is the overall average. However the price for unrealistic request 
is dismissal of the monotonic pattern from Fig. [IJc): instead, the robot spirals along a spring-like path, like in 
Fig.HDb). Such path can be viewed as the result of a periodic motion along a simple closed curve in a clockwise 
direction, which motion holds in a reference frame that is translating at a constant speed rc > 0 in the gradient 
direction ng := (cos sin t/?)^. 

Lemma 3.1. Suppose that the requested field growth rate is unrealistic v > vn. Then the closed-loop trajeetory 
r[t), including the velocity orientation angle 9(t), has the following form 

Stt 

r{t) =rQ{t-to)-\-r.^ + {t-to)wno: 6>(t) = - tg)- (t-to). (3.5) 

T 

Here 0o(‘) : R —^ R and r 0 {-) : R. —>■ are periodic functions with a common period r > 0, which along with 

the speed w > 0, do not depend on the robot’s initial location Tin, whereas the constants tg G R and r* £ R^ 
are determined by Tin. The equation r = rQ{t) describes a motion along a simple closed curve in the clockwise 
direction, and the robot’s velocity constantly rotates clockwise: 0 < 0 . 

Proof: Like in the proof of Theorem 13.11 we assume, without any loss of generality, that = 0 in (13.IL We 
also put 0* := arccos It is easy to see that now (EH) should be replaced by 

0 = fj,[vncos0 — z^] = —~ cos0cos0*] < — cos0*] < 0. (3.6) 

So any solution 0{t) goes from -|-oo to —oo as t runs from —oo to oo and is a time-shift 0{t) = 0o(t — to) of 
the unique solution 0<>(-) with 0<>(O) = 0. Here 9o{—t) = —0o(t) since 0{t) := —0(^{—t) clearly solves (13.61) and 
0{O) = 0. Furthermore, 


0{t -I- r) = 9{t) — 2tt, where r := 


dO 


1 


yLv 1 ~ cos 0 cos 0* 


> 0 . 


So the function 9o{t) := 9(){t) ^t is r-periodic. Hence the second relation from (13.51) and the last claim of 
the lemma do hold. Taken 9 as an independent variable, we have by (EH) and (EH), 


dx 


COS0 


d9 


sin6( 


pLu 1 — cos 0 cos 0* d0 pLvl — cos 0 cos 0* 

Let a;*(0) and ?/*(0) be the respective solutions with x*(0) = 2/*(0) = 0. Since sin and cos are odd and even. 
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respectively, 


/■ 


sin0 


1 — cos 6 cos 


de = y*(6»+ 27r) = y*(6»); y*(-6») = y*(6»), y*(6») < 0 V6» € (0,7r); (3.7) 


cost 


1 — cos 9 cos 0* 

COS0 


1 — COS 0 cos 6 


d9- 


COSf 


r 1 — COS 9 COS 9t 
cos 9 

Lil 1 + cos 0 cos ( 


d9 = 


cos 9 


1 — cos 9 cos 0. 


d9- 


cos{9 + tt) 


j_ 2 L I — cos{9 + tt) cos 9^ 


d9 


d9 = 2 


cos^ 9 cos 0* 


_2L (1 — COS0COS0*)(1 + COS0COS0*) 


d9=:L>0 


a:*(0 + 2tt) = x^,{9) - —L; x*(0) = -a;*(-6»). (3.8) 


Hence the function a;**(6>) := a;*(6') + is 27r-periodic. Soa;o(t) := a;**[6»o(<)] - -0ft 

and ?/o(t) := y*[0o(t)] are r-periodic and the first relation from (13.51) holds with w := It remains to show 
that the curve with the parametric representation x = XQ(t),y = yo{t) does not intersect itself during its 
period t S (—t/2,t/2). Suppose to the contrary that such intersection occurs. By equating the respective 
y-coordinates with regard to the last two relations from (1X71) . we see that this intersection may hold only at 
symmetric times t = ±to € (—r/2,T/2). Then the respective x-coordinates are different since X 0 (-) is odd by 
(ixa . This contradiction completes the proof. □ 

The last claim of Lemma 13.11 entails that the velocity orientation angle 9 arrives at values of the form 
ip + ip + 2'iTk, with ip = 7r,7r/2, —7r/2 and an integer fc, every r units of time. At the respective time instants, the 
robot moves either in the anti-gradient direction or perpendicular to it; see Fig. [3Da). Thus unrealistic request 
u > vn results in motion that involves systematic sideward and backward maneuvers and so is rather ineffective, 
despite overall gradient climbing. 

By the foregoing, the parameter p > 0 of the controller (EH) influences neither convergence nor behavioral 
pattern. To illuminate the role of /x, we focus on the case of a realistic request v < vn, fix initial location 
rin, and examine various y > 0. Let r^(-) and correspond to the respective closed-loop trajectory. Since 
9{t) := 9i{t) satisfies equation (13.41) of the closed loop system, where y := 1, it is easy to see via the change of 
the variables t := ^r',t := yt' that 9{t) := 9i{iJ,t) solves (13.41) with y at hand. Thus = 9i(pt). It follows 
that increase of y proportionally shortens both time and length of the transient to the beneficial equilibrium 
9 = 9^, where the requested field growth rate is achieved. Indeed let to be the time for which 9i{-) reaches the 
e-neighborhood of 9^, i.e., \9{t) — 9^\ < e'it > to- Then 9^{-) = 9i{y.t) evidently does this y times faster, i.e., 
satisfies the same inequality for t > tg := During the transient to that neighborhood, the deviation of 

r^(-) from the initial location 


max ||r^(t) 
te[o.t^| 


- r;, 


V max 
se[o.to| 



e[9f^{T)] dr 




V max 
se[0.io| 



e[9,{0]d( 


= y ^ max ||ri(s) - rin|| 

s6[0,to| 


is /i times less than that for Ti(-). Thus increase of y shortens transients in both time and space. This is of 
interest for generic fields with varying gradients. Large enough y gives promise to convert transients into so 
“fast” and “short” motions that they are negligible as compared with the gradient change rate. Since after the 
transient, the fields value ascends, this should ensure successful overall gradient climbing. 

Thus we have acquired first evidence that tuning of the controller (ESI) should follow two guidelines: firstly, 
the requested field growth rate v should be realistic and secondly, the parameter y should be large enough. To 
flesh out, specify, and justify this, we proceed to theoretical analysis of the closed-loop behavior in a generic 
smooth dynamic field D{-). 


4 Convergence to the Maximizer in Generic Dynamic Fields 

We start with notations, which are mainly concerned with characteristics of dynamic fields. 


4.1 Basic notations and field characteristics 

• V = f (1; ) — the spatial gradient; 

\ dy J 

• D" — the spatial Hessian, i.e., the matrix of the second derivatives with respect to x and y; 
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Figure 3: Two close isolines. 


• I(t, 7 ) := {r : D{t, r) = 7 } — the spatial isoline, i.e., the level curve of D{t, •) with the field level 7; 

• [T,iV] = [T{t^r),N{t^r)] — the (right) Frenet frame of with 7 := D(t,r) at the point r, i.e., 

N = ||y^p’r)|| T are the unit normal and tangent vectors, respectively; 

• K = >c{t, r) — the signed curvature of the spatial isoline; 

• r+(At|t, r) — the nearest (to r) point where the ordinate axis of the above Frenet frame of the isoline 
I{t, r) intersects the isoline with the same field level 7 observed at time t = t + At; see Fig. [Sj 

• p(At|t, r) — the ordinate of r+(At|t, r), i.e., the final normal displacement of the isoline from t to t + At; 

• A(t, r) — the front velocity of the spatial isoline: A(t, r) := limAt-i-o ’ 

• a{t,r) — the front acceleration of the spatial isoline: a(t,r) := limAt-s-o ■ 

• Aip{At\t,r) — the angular displacement of T[t+ At, r+(At|t, r)] with respect to T[t,r]; 

• — the angular velocity of rotation of the spatial isoline: U!{t,r) := ^im — 


A 7 


-, where g(A 7 |t, r) is the ordinate of the nearest 


p(t, r) — the density of isolines: p(t, r) := lim ,, , , , 

^ ' A7^og(A7|t,r)’ 

to r point where the ordinate axis of the Frenet frame intersects the isoline /(t|t, 7 + A 7 ), 7 := D(t, r); 0 


Vp{t,r) — the (proportional) growth rate of the density p over time: 


At.'r) ■= 


1 


, , lim 

p{t,r) At-s-o 


p[t + At, r+ {At\t, r)] - p[t, r] ^ 
At 


Tp{t,r) — the (proportional) growth rate of the density nnder a tangential displacement at time t: 


Tp(t,r) := 


1 


/ N lim 
p[t,r) As-s-O 


p{t,r + TAs) - p{t,r) 
As 


np{t,r) — the (proportional) growth rate of the density under a normal displacement at time t: 


o{t,r) := 


1 


lim 


p{t,r + NAs) - p{t,r) 


p(t,r) Ab^o As ’ 

• — the angular velocity of the gradient VD rotation at time t at point r. 


(4.1) 


(4.2) 


(4.3) 


The following lemma explicitly links the above quantities with D[-). Further (•; •) is the standard inner product. 

Lemma 4.1. Whenever the field D{-) is twice continuously differentiable in a vicinity of {t, r) and VDit, r) 7 ^ 0, 
the afore-introduced quantities are well-defined and the following relations hold at {t,r): 


A = - 


D' 


{VD[ + \D"N- N) 


iViAll 


p = II ViA||, r+(dt|t, r) = r + \Ndt + o{dt) 

— Xv 


iiviAir 

(VD( + AD" A; T) 


D"+A(VD(;A) 


(D'T; T) 

IlVDil ’ 


wv = — 


l|VD|| 
IVDII 


l|VD|| 


P5 


To = 


{D"N- T) 

IVDII ^ 


Uo = 


{D"N- N) 

IVDII ■ 


(4.4) 

(4.5) 

(4.6) 


^This density characterizes the “number” of isolines within the unit distance from the basic one 7(t, 7 ), where the “number” is 
evaluated by the discrepancy in the values of D[-) observed on these isolines at the given time t. 
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Proof: The first claim and (14.41) are immediate from the implicit function theorem [5D]. We put r+{At) := 
r+{At\t,r). Due to (14.4L 

VD [t + dt, r+{dt)] = VD[t + dt,r + XNdt + o{dt)] = VD + + XD"N]dt + o{dt), 

p[t + At, r+(dt)] = \\X/D [t + dt, r+((it)]|| = || VD + [VDj + XD"N]dt + o{dt)\\ 


{X/D-VD'+XD"N) , 

— Il^^ll ^ -ijVDjl- 

= p + {N-,VD[ + XD"N) dt + o{dt) |=^ the first formula in (14.5|) . 


Arr . /.M VD [t + dt, r+((it)l 

[t + dt, r+(dt) = 11^ . -;- , , = N ■ 

^ ’ +v tJ ||VD[t + dt,r+(dt)]|| 


VD[ + XD"N 


VD 

iivDir 


{\JD-,\JD[ + XD"N) 


(4.7) 
dt + o{dt) 


= N+ [VD( + XD"N - N {N; VD[ + XD"N)] dt + o{dt) N + + o[dt), 

where (a) holds since W = (W,T)T + (W, N)N VW G K^. In the Frenet frame {T,N), we have 




= N- 


— cosO 

— sinO 


ujdt + o{dt) = N — LoTdt + o{dt). 


By equating the coefficient prefacing Tdt in the last two expressions, we arrive at the second formula in (14.51) . 
Finally 




\\X/D{t + dt,r+{dt))\\ 


EH, D" +A(VD(;iV),^ ^ 


l|VD|| 

(iV; VD( + XD"N) 
IIVDII 


||VD||2 

dt + o{dt) X — dt — Xvpdt + o{dt). 


where (b) follows from the first formula in (14.51) . The definition of a completes the proof of (14.51) . The first two 
equations in (ITO are well known, the third one follows from the transformation 

{D"T-VD) 

p{t, r + Tds) = II VD[t, r + TdsJII = pit, r) + + {D"T-,N) ds + o(ds), 

the fourth equation in (14.61) is established likewise. □ 

It follows from Lemma lO that UJ = UJy — XTp, Vp = —UJy + XUp. 


4.2 Assumptions Underlying Theoretical Analysis 

In the most general setting, the problem at hand comprises problems of global optimization. In the presence 
of local extrema, NP-hardness, this mathematical seal for intractability, was established for even the simplest 
classes of such problems m- Since we do not mean to cope with the challenge of NP-hardness, it is permissible 
to examine the closed-loop behavior in a field with no local extrema. So our first assumption considers a smooth 
held with a single global spatial maximizer r°(t) and no local extrema, which held converges to a hnite limit 
7 inf(t) as ||r|| -)> oo. 

Assumption 4.1. The field Df) is C^-smooth, and there exist continuous functions r®(t) and 7 inf(d < 
D[t, r°(f)] such that VD(t, r) ^ 0 Vr ^ r°(t) and D{t, r) —>• 7 inf (i) as ||r|| —>• oo. 

Hence maXrgR 2 D{t,r) is attained at r = r°(f), and for given t, the held value D{t,r) ranges from 7 inf(d 
to D[t,r^if)\. Assumption 14.II will be relaxed in Remark |4.II 

For steady helds, no other assumptions are imposed. So the remainder of this subsection addresses only 
unsteady helds. 

If during the search, its conditions, characterized by the held parameters, vary over a very wide, up to 
inhnite range, regular closed-loop behavior can hardly be ensured by a common set of controller parameters. So 
we will try to inclose the path of the robot in a region within which the held parameters from (14.41) — (14.61) 
are bounded over space and time. This motivates us to discard locations that are excessively both distant and 
close to r^if), with the latter being unwelcome since, e.g., the isolines typically become highly contorted >c ^ oo 
when approaching the maximizer r^ff). To reduce technicalities, we pick the region Zreg so that it is upper and 
lower bounded by some isolines I\t, 7 -(t)] and I[t, 7 +(t)] whose levels 7 +(t) > 7 -(t), 7±(0 ^ ( 7 inf( 0 j 
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may, however, evolve over time. The role of Zj-eg is to accommodate the transition from rin to the desired i?*- 
neighborhood Vi,(t) of the maximizer. So we pick 7 ±(-) so that the region links and 14(t). Summarizing, we 
arrive at the following choice 

Zreg ■■= {{t,r) : 7 _(t) < D(t,r) < 7_(0) < D[0,rin\, 

7 i„f(t) < 7 _(<) < 7 *(t) < 7 +(t) < D[t,r°(t)], (4.8) 

and an intermediate level 7 *(t) is picked so close to r°(t) that 

D{t,r) > 7 *(f) =7 r e 14(0- (4-9) 

We recall that 14 (t) is the desired vicinity of the maximizer where the robot should be ultimately driven. 

For steady fields, the levels 7 i(-) are chosen constant, Z^eg has the form C x [0, oo), with C compact, and so 
all field parameters from (ITO are bounded over Z^eg; their upper bounds will be of interest for controller 

tuning. For time varying fields, they are bounded over any finite time interval by the same argument. The next 
assumption is non-void only for dynamic fields and claims that boundedness does not degenerate as t ^ oo, 
which is needed only because the subsequent analysis is asymptotic and so assumes that t does go to oo. 

Assumption 4.2. There exists constants for H = p,X,uj, >c,v,a,n,T and 7 +, 7 such that 

\p\ < bp, |A| < 6 a, IwI < bu,, |wv| < b^, 

\^\ < b^, \vp\ < by, |a| < ba, \Tp\ < by. 

To advance to the maximizer, the robot should transverse isolines. So it is natural to assume that its speed 
exceeds that of isolines u > |A|. A particular isoline with the time-varying level 7 = 7 *(t) characterizes the 
desired vicinity of r^{t) within which the robot should ultimately be kept. So it is also reasonable to assume 
that the mobility of the robot exceeds that of this isoline. Since the front speed of the latter is A -I- p“^ 7 *, which 
can be shown similarly to (14.41) . we assume that w > |A -I- p“^ 7 *|. Finally we enhance the both assumptions by 
unifying them into the requirement rJ > |A| -I- p ~^7 imposed in Z^eg- It is always satisfied for steady fields since 
then A = 0 and 7 = 0 . 

Finally, all assumed strict inequalities are protected from degradation as t —>■ 00 : 

p{t,r)©Q, V ©\X{t,r)\ + in Z^g; 7 _(t) © 7 *(t) © 7 +(t) in[0, 00 ). (4.11) 

Here the notation f © g in Z is used to express that there exists e > 0 such that f{t, r) > g{t, r) -|- e for all 
{t,r) G Z; the relation © is defined likewise. In (I4.11L the first inequality protects the requirement p{t,r) = 
||ViA( 4 r)|| > 0 from Assumption 14.11 whereas the last protects the inequalities 7 -(t) < 7 *(i) < 7 +(i) from 

(HU). 

For steady fields, Assumption 14.21 and (14.1111 are automatically fulfilled. It follows from Lemma im that for 
generic unsteady fields satisfying Assumption 14.11 and (14.lip , boundedness of the field parameters requested in 
the first row from (14.101) holds whenever the first and second derivatives of the field are bounded on Z^eg. 

4.3 Main Theoretical Results 

Our first theorem provides an evidence that the proposed control paradigm (12.21) is intrinsically capable of 
extremum seeking in an understandable context introduced in the previous subsection. 

Theorem 4.1. Suppose that (14.1111 and Assumvtions W?\[ 14.21 are valid. Then there exist parameters v,p,,d.f of 
the controller such that the following claim holds: 

(i) The controller ( 1 ^ brings the robot to the desired vicinity Vi, ft) of a maximizer in a finite time to and 
keeps it there afterwards: r{t) G Vi,(t) for t > to- 

Moreover, for any bounded and closed domain D lying in the interior of the set {r : (0,r) G Zy^g}, there exist 
common parameters zz, for which (i) holds whenever the initial location Tin G D. 

The proofs of all claims from this section are given in Section [51 Now we proceed to discussion of controller 
tuning. Its parameters d* and v are chosen prior to /i. Whereas d* is arbitrary, v is chosen so that 

V©\X\+P~^ 1 ' in Zreg, iz > q. (4.12) 

This is feasible by (14.lip . For steady fields, A = 0 ,7 = 0 and so (14.121) takes the form f7|| ViA(r)|| > 0 V(t, r) G 
Zreg by (14.41) . This conforms to the first guideline stated at the end of Sect. [3] the requested field growth rate 


^(^5 ^ Zyyg', 

l+{t)<l%, l7i(i)|<7 'it,i = ±,-k. (4.10) 
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V should be realistic, i.e., lie in the interval d = {v\\7D) G [—ti||VD(r)||, u|| VD(r)||], where the equation holds 
by (EH). It can be shown that the first relation from (14.1211 has the same meaning for dynamic fields as well. 
By the second and last inequalities in (14.1211 and (I4.10|l , respectively, the requested field growth rate ^ exceeds 
the growth rate %(t) of the isoline level 7 *(t) that is associated with the desired vicinity of the maximizer. 

By the second of the above guidelines, /i should large enough. To specify this, we pick lower estimates 
Av,A.y > 0 of the discrepancies in the first, third, and fourth inequalities from (14.1111 and the first strict 
inequality from (113); 


p(t,r)>Av V(t,r)GZreg, j-(t) + Aj < < j+(t) - Aj, n[0,rin] > j-(0) + Ay/2. 


(4.13) 


After invoking the quantities from ()4.10ll . the choice of p is specified as follows: 


/i ® ■ 


z/ — 7 




npl 


vtP vt vtP 


where Vt := ±\/f^ — [A + in Zi.eg;(4.14) 

in ^reg;(4.15) 


p{v — X) — V 


u> - min max{ai(fe), a 2 (fc)|, where 

1/ — 'y k=l,2,... 


ai{k) 26|^ [l + i] + + b1 [2 + A] 

n,\ _ OV ( 2 fc+l)i 7 + 27 i-(fc+l)(b>+A~^ 7 ) 

a2\K) ■— ^Op ^ 


.(4.16) 


Since ai{x) and 02 ( 2 :) are descending and ascending functions of a: > 0, respectively, min^ in (14.161) is attained at 
the integer either ceiling |"a:„iinl or floor [a:„iinj of the positive root Xmin of the quadratic equation ai(x) = 02 (x). 
Now we show that under the recommended choice of the parameters, the control objective is achieved. 

Theorem 4.2. Suppose that (14.111) and Assumvtions WH 14.21 are valid, and the controller parameters satisfy 
(I4.12P and (I4.14|) — (14.1611 . Then (i) from Theorem 14.11 is true. 

Choice of v, p satisfying (14.1211 . (14.1411 . (14.1511 may proceed from estimation of the respective right-hand sides, 
based on estimates of the field parameters. For example, (14.1011 and (14.1311 ensure (I4.12|l . (14.1411 . (14.1511 whenever 


p > 


1 


V — ^ 


v>hx + —, 

Ay 


bui + br{b\ - v) 

Av(v - bx) - ly’ 


2b^ bj^v -f 2 


brl 


-0 2 


by-f 
Vt Ay 



Vt 


bnl"^ 

vtA%_ 


/ 

r — 1 

where vt := ± 1 / v^ — 



If the isoline levels are constant, 7 = 0 and so all small enough v and large enough p obey the bounds (14.1211 . 
(I4.14|l (I4.16|l from Theorem 14.21 This may be used as a guideline for experimental controller tuning. 



Figure 4: Subdomains of the workspace. 


Remark 4.1. Theorems 14.11 IT^ remain true if (14.1111 and Assumption 14.21 are still stipulated, whereas As¬ 
sumption O is relaxed so that the field may have local extrema and be non-smooth in a vicinity .Znear of the 
global maximizer and in an outlying region Zfar; see Fig. U] Instead of Assumption 14.11 it suffices to assume 
that D{-) is continuous and there exist smooth functions 7 -(t) < 7*(t) < l+{t) G R of time t such that the 
following statements hold: 

i) On the set Z^eg := {{t,r) : 7 _(t) < D{t,r) < 7 +(t)}, the held distribution D{-) is identical to a C^-smooth 
function dehned on a larger and open set, and has no critical points VD(t, r) ^ 0; 
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ii) The set {r : 7 _(t) < D{t,r)} is bounded for any t and D[0, Tin] > 7-(0) ; 
hi) The implication (14.91) is true. 

Moreover these theorems remain true even if the regions Znear ■= ■ D{t,r) > 7 +(t)} and Zf^i ■= {{t,r) : 

D{t,r) < 7 -(t)} contain point-wise singularities where the field is not defined. 

For continuous fields, max.rgR 2 r) is attained and all global maximizers lie in 14(t) due to i)—hi). 
Thanks to the absence of critical points, the region contains no local extrema, whereas they may lie in 
ZneaY and Zfar- 


5 Illustrative Examples 

Now we complement the previous discussion in simple yet instructive cases of special and not completely known 
moving helds. We show that any of these fields satisfies the assumptions of Theorems 14.11 and 14.21 and provide 
an explicit description of controller parameters {v, n) that guarantee convergence to the Held maximizer. 


5.1 Radial Field with a Moving Center 

We consider a scalar field of the form D{t,r) = c/(||r — r°(<)||), where c > 0 and 'r°(-) are unknown, whereas 
the twice continuously differentiable function / : [0, oo) —>■ R is known, strictly decaying and convex f'{z) < 
0, f"{z) > 0 Vz > 0. An example of such field is minus the distance to an unknowingly moving target 
D{t, r) = —||r — r°(t)||, in which case c = 1, f{z) = —z. Other examples are concerned with some fields caused 
by emanation of a certain substance from an unknowingly moving point in a homogeneous unbounded medium. 
If the speed of this point is sufficiently small so that the process is well approximated by the time succession 
of the instantaneous equilibria, the mathematical model of the field may be given by the above formula, where 
the unknown c characterizes the emanation rate. 

The objective is to display the unknown center r°(t) of the field by bringing the robot to its i?*-neighborhood 
{r : ||i— r°(t)|| < i?*} on the basis of the following known estimates 

Ikin - r°(0)|| < i?in, ||f'°(t)|| < ?;o, ||r°(t)|l<ao Vt > 0, 0 < c_ < c < c+. (5.1) 


We assume that Rm > R* to simplify the formulas. 


Proposition 5.1. Let the robot be faster than the field center v > vq, hold, and the parameters v, p, of the 
controller (1^ be chosen so that for some R- > Rm, 0 < i?+ < i?*, the following relations are valid: 


L > 


4vo -I- 6t; 
vR^ 


pL> V 


+°o v + 2vo 

\/v'^ - vl R+ 


0 < 1 ^ < c-\f'{R-)\{v - Vo), 

_1)0_ 

R+ c-\f'{R-)\{v - Vo) - i/’ 


^,>2^\f{R.)\ 

V 


+ 47^^Jo 

min{/(i?*) - /(i?_);/(i?+) - /(i?,); 2[/(i?„) - /(i?_)]}' 


(5.2) 

(5.3) 

(5.4) 


Then the controller (1^ brings the robot to the desired R^,-neighborhood of the center r^{t) in a finite time to 
and keeps it there afterwards ||r(t) — r°(t)|| < i?* Vt > to- 

The described choice of v and fi is possible since in (15.21) — (15.41) . all right-hand sides are finite and positive. 
Proof of Proposition [57ll We take 7 i(t) = cf{Ri),i = *, ± in (14.81) and invoke that $.|iV is the counter¬ 
clockwise rotation through the angle 7r/2. An elementary calculus exercise based on Lemma [4.11 shows that 


N = 


T = -^^N, A = (f°;V), p = c\f\, Tp = 0, = JJTy 

= 0> w = wv = - iit’jLl ’ « = 


,011 ■ 


Now Z^eg = {{t,r) : R+ < ||r — r°(t)|| < i?_} is a time-varying annulus and so in (j4.10p . (I4.13|) . 


7 = 0, 

Av=c_|/'(i?_)|, 


bu, = hZ = b^ = -^, br = 0, bp = c+\f'{R+)\, bx = vo, 

= c_ min{/(i?*) - /(i?_); f{R+) - /(i?*); 2 [/(i?in) - fiR-)]}- 
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Hence the assumptions of Theorems 14.11 and 14.21 are fulfilled in the form given by Remark 14.11 whereas (14.121) 
holds by (EH). Inequalities (14.1411 and (14.151) take the form 


© 




yj v'^ — 


(Xlll 1 

r —r^ll’ ||r — r^ll p(z; — A) — 1 / 


and follow from (Eni) since takes values in {—y/v^ — A^, \/vq — A^] for given A. Relations (14.1611 with 

fc = 1 are implied by dEl. Theorem 14.21 and Remark 14.11 complete the proof. □ 

Now we discuss in more details a particular case of the problem at hand. 

5.2 Locating and Escorting a Moving Target Based on Range-Only Measurements 


In this case, D\t,r\ = — ||r — ' 


, c = c_ = c+ = 1, f{z) = —z, and the field center r°(t) is at the target. 


Proposition 5.2. Let the target r^{t) and the robot initial location Tin satisfy the first three inequalities from 
(EU, and the robot be faster than the target v > vq. Then the conclusion of Proposition 15.11 remains true if 


0 < u < V — vq 
> max < 


2vo 


12v + 8^0 IbTrtio + 12r; 


Oq 


R^(v-vo-iy)’ 

Proof: Now (15.211 — (15.411 take the form 

ao 


vR* 


'\/v^ - V, 


+ 


0 ' 


V + 2vo 


0 < ly < V — Vq, yiv > 


V + 2z;q + 


4r;o + 617 

yLV > ---, pLV > 2- 




T > 


1 


^/v'^ -Vf 


Vo 


(5.5) 


i?+ V — Vo — 


3v + 47ruo 


R+ ’ ^ min{R_ — R*; R* — R+; 2(R_ — Rin)} 
The freedom to arbitrarily pick R_ permits us to let R_ —>■ oo here, which results in 

_ ^0 + V + 2vo (a b 

V < v < V — Vo, pLv > = H- - -, pL > max • 


\Jv^ 


Ro 


R+ R* ~ R 4 


where a := max ^'^ 0 + 61 ; | ^ ^ 2 3'»+4'^'»o ^ gy picking R_|_ := R*/2, we arrive at (15.51) . Proposition 15.1 

completes the proof. □ 


6 Proofs of Theorems 14.IL 14.21 and Remark 14.11 

We start with a purely technical exercise in calculus, which addresses quantities introduced in subsect. 14.11 
Lemma 6.1. The following relations hold: 

X{t,r + Tds) = X + aids + o{ds), X{t,r + Nds) = X — Vpds + o{ds); (6.1) 

N[t,r + Tds] = N — nTds + o{ds), T[t,r + Tds] = T + xNds + o{ds); (6-2) 

N{t,r + Nds) = N + TpTds + o{ds), T{t,r + Nds) = T — TpNds + o{ds), (6-3) 

N[t + dt,r^{dt)] = N — (jjTdt + o{dt), T[t + dt,r+{dt)] = T + wNdt + o{dt), (6.4) 

where r+(At) := r+(At|t, r). 

Proof: Formulas (16.11) are justified by the following observations: 


X{t,r+Tds) ^ - 


D'ft, r + Tds) 
\\VD{t,r + Tds)\\ 


= Ht,r)- 


^,{D"N-T) 


IVRII 


-ds+D 


IVRII 


ds+o{ds) X{t,r)— 


IIVRII 


ds 


(D"N;T) ^ ,, , (VRI( + AR»" A; T) ^ ^ EU ^ 

— A ' ds + o{ds) = A(t, r) - .. .. - -ds + o{ds) = X + ujds + o{ds); 


IVRII 


IVRII 


(VR)( + XD"N; N) 


ITsl 


A(t, r + Nds) = X{t, r) — —— * |'|VR)|| —^~ 
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Formulas (16.21) are the classic Frenet-Serrat equations. Furthermore 


^ VD(t,r + Nds) 

N{t, r + Nds) = _: ’ -= N 


= N 


||V£»(t,r +Afds)|| ||Vi:»|| 

D"N - N {D"N-,N) 


D"N ^ ^ID"N-VD), 

ds — \7D-—ds + o{ds) 


llVi^lP 


IVT^II 


ds + o{ds) = N + Tds + o{ds) N + TpTds + o{ds), 


which yields the first formula in (16.3L This also implies the second formula since N = ^jlT^T = —^jlN. 
Formulas dMl) follow from the definition of w. □ 

From now on, we examine the robot driven by the control law (12.2F Assumption 14.11 is taken in the 
generalized form, i.e., as i)—iii) from Remark |4.II Prior to analysis, we note that thanks to (I4.12p . (I4.14p . along 
with (I4.10L (14.1111 . there exist 

(5€(0, — 7 ), 5 ~ zz — 7 (6.5) 


and Si > 0 such that everywhere in .^reg, 


( 6 . 6 ) 


> |A| + UA + El, where va ■= p (i^~<5), 
pS >-2uj - xv't + 2TpVA + —^ + np^+e 2 , where := ±\Jv'^ - [A + va]^, (6.7) 


and (14.161) holds with i5 put in place of zz. We also observe that the closed-loop system obeys the equations 


dD. 


r = ve{d), 0 = p[d-iy], d = v {VD{t,r)-,e{9)) + 


( 6 . 8 ) 


The next two lemmas display the key features of the control law (12.211 . The first lemma shows that under certain 
conditions, d = v — 5^d>0. This implies that d cannot trespass v — 5 from above and so if the inequality 
d>v — 5 is achieved, it will be kept true afterwards. By (14.1011 and (16.511 . this implies that d> | 7 *| and so d{t) 
not only ascends but also eventually overtakes 7 *(t). The last means that the robot enters the desired vicinity 
of the maximizer by (SH). 

Lemma 6.2. For all (t,r) G Zreg, the following implication holds 


d = V — 5 


d>0 ifcr := (T;e( 6 l)) < 0 
d <0 if O’ > 0 


(6.9) 


Proof: With start by noting that 

V — 6 = d = D't+v <VD,e>^^ P (—A-I-17(iV, e)) => (A^, e) = 3 -I-where va'■= P~^{v — 6), 

V V 


1 r 


=> e = 


(A-I-ua) + sgncry - [A-I-ua] T 


( 6 . 10 ) 


r{t + dt) = r{t) + vedt + o{dt) = r{t) + XN dt -\-[vaN + VTT\dt + o[dt)-, 

^ ^ > 

—rj^{dt)-\-o{dt)hy (14.41 

d{t + dt) — d(t) = p[t + dt, r{t + dt)] (—A[t + dt, r{t + dt)] +v {N[t + dt, r(t -f dt)],e(t + dt))) 

-P[t, r{t)] (-A[t, r{t)] + V (N[t, r{t)],e{t))) 

= {p[t + dt,r{t + dt)] - p[t,r{t)]} {-X + v{N,(^) -p{X[t -b dt,r{t + dt)] - A[t,r(t)]} 


( 6 . 11 ) 


+vp(^ A^[t -b dt, r{t + dt)] — N[t, r{t)],e) + vpi^N, e{t + dt) — e{t) ) -b o{dt); 


cl 


c2 


a {p[t + dt, r+{dt) + VANdt + vxTdt -b o{dt)] — p[t, r(t)] p[vp -b UA^ip + VTTp]dt -b o{dt). 

Similarly and with regard to the definition of the front acceleration a, 

(I6.lt r 1 7, / T,\ 1 (I6.2t - (l6.4t . , . , 

0 = [a -b lvvt — VpVA]dt -b oydt), cl : [—w -b TpVA — >cvt]-L dt + 0{dt), 

c2 ^ ei^i^edt + o{dt) ^ = [- (A -b va) T + vtN] dt + o{dt). 

2 y 
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Summarizing with regard to (16.101) and the expansion d{t + dt) — d{t) = d{t)dt + o{dt), we see that 
d{t) = p[{vp + VAnp + VTTp)vA - (a + ujvt - VpVA) + (-w + TpVA - >cvt)vt + vt0\ 

T /i f \ \ d—v (5 f. 

where a = /r(a — v) —po. 

The proof is completed by noting that the expression inside [...] does not exceed —ei by (EH), and vt 
v{T;e). □ 

By Lemma |6.21 the key inequality d > v — S is maintained whenever cr < 0. The next lemma implies that 
while the inequality is true, a cannot trespass zero from below and so the condition cr < 0 is kept true. 

Lemma 6.3. For a{t) = {T[t^r{t)];e[6{t)]), the following implication holds 

a = Qf\d>v — 5^a<Q. 


= pVT 


9 — 2uj — >cvt + ‘2.TpVA + 2r;p —- 

Vt Vt 


Vt 


Proof: We first note that cr = 0 ^ e = ±N. If e = —iV, we have 


(im i iTTO 

d = Df. + V (VU; e) = D[ — vp = —p(A + n) < 0, 
in violation oid>v — 6>0. Hence e = N. Similarly to (16.lip . 

r{t + dt) = r{t) + vedt + o{dt) = r{t) + vNdt + o{dt) = r+(t) + (U— X)Ndt + o{dt), 

T[t + dt, r(t + dt)] — T[t, r{t)] [w + rp(A — v)]Ndt + o{dt), 

e[0{t + dt)] - e[9{t)] ^ 0<^>^mt + o{dt) = 9(^>^Ndt + o{dt) = -0Tdt + o{dt), 
a{t + dt) — a{t) = {T[t + dt, r(t + dt)] — T[t, r{t)];e) + {T-,e[0{t + dt)] — e]9{t)]) 

= ([w + Tp{X — v)]N;e) dt — (t;0t'^ dt + o{dt) = [w + rp(A — v) — 0]dt + o{dt); 

d = D[ + V {VD-, e) = D[+vp = piv — A); 0 p{d — v) = ppiv — A — v/p)', (6-12) 

_ _ ITTsI 

a = uj + Tp{X — v) — pp{v — X — V / p) <0. □ 

Up to minor details, the next claim shows that the robot reaches the desired vicinity of the maximizer 
provided that initially the afore-mentioned inequalities d{t) >v — 5 and aft) < 0 hold. 

Corollary 6.1. Suppose that dft) > v — S,a(t) < 0, and [t,r{t)] G Z^eg at t = tg. In a finite time, the robot 
reaches the curve /[t, 7 +(t)], with maintaining the above inequalities and inclusion true. 

Proof: If r{to) £ /[to, 7 -i-(^o)], the claim is trivial. Otherwise, D[t,r{t)] < 'y+{t),d > re — d > 0 at t = to, 
and d{to) = v — 5 ^ d{to) > 0 by Lemmawhereas cr(to) = 0 => cr(to) < 0 by Lemma HOTl So the open set 

S' := {t > to : 7 -(t) < d{t) < 'y+{t),d{t) > v — 6, aft) < 0} 

contains an interval of the form (to,t*),t* > to. Let us consider the maximal of such t*’s. Here t* ^ oo since 
otherwise 7 +(t) > d{t) > d{to) + (i/ — S)ft — to) —> oo as t —> oo, in violation of the penultimate inequality from 
(14.101) . Hence t* < oo and either (a) 7 -(t,) = d(t,) or (b) cr(t*) = 0, or (c) cr(t*) < 0 and d(t*) = v — 5, or (d) 
d(t*) = 7 +(t*). However (b) and (c) are impossible by Lemmas 16.31 and respectively, whereas 

1631 14.101 

d{to) > ^-{to) A d{t) > n — 6 > 7 > 7 _(t) d(t*) > 7 _(t*) 


and so (a) does not hold either. Thus (d) holds, which completes the proof. □ 

Now we show that even if the above inequalities dft) > v — 5 and aft) < 0 do not initially hold, they become 
true at some time instant. To this end, we need some technical facts concerned with motion in the situation 
where some of them is violated. 

Lemma 6.4. Suppose that dft) < v — 5, [t,r{t)] £ Zreg Vt £ A = [to,ti]. For t £ A, the vector e{0) rotates 
cloekwise with the angular velocity 0 < —pS, and the space deviation from the initial location obeys the inequality 


2v 


r{t) - r{to)]\ < q{(p) ^ ^ ^ [1 - cosmin{^ 7 )]; tt}] 


pS L27rJ pS 


where p := ]9{t) - 9{to)], := p - 2Tr . (6.13) 
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Proof: Without any loss of generality, we assume that to 

lemma is immediate from the second equation in (16.81) : 9 = ^ 
new independent variable: 


d 


0 ,r( 0 ) = 



d<L/—5 

< 


0,0(0) = 0. The first claim of the 
—^S. So s := —0 can be taken as a 


dr V 

— = ue(—s), u := —— € 

ds 6 


V 

.V. 


The squared distance ||r((^)p does not exceed the maximal value of I in the following optimization problem: 

d'P 

I := | 1 t'((/ 3 )||^ —>• max subject to '^ = ue{—s) s £ r(0) = 0, u{s) G [O, . 

Its solution r°(-), M°(-) exists and obeys the Pontryagin’s maximum principle [33]: there exists a smooth function 
'(/'(s) G such that 


u°{s) = argmax„g[o_^/(^ 5 )]uV'^e(-s) 


= 


v/{fi6) 

0 


unclear 

\ 


if ip^e{—s) > 0 
if ^^e(—s) < 0 , 
if ' 0 ^e(—s) = 0 


where 


f =-|:nV^e(-s) 


0^4’ = const 


If r^{ip) = 0, (16.1311 is evident. Let r^{(p) 0. Then tp ^ and so as s progresses, u°{s) interchanges the values 

0 and v/{fi6), each taken on an interval of length tt possibly except for extreme intervals whose lengths do not 
exceed tt. Inequality (16.1311 results from computation of ||t’((^)|| for such controls u(-), along with picking the 
maximum among these results. □ 

The next step is a technical study of isolines with time-varying field levels, like those bounding the desired 
vicinity of the maximizer and the operational zone by i)—iii) of Remark 14.11 For any simply connected 
domain 2) C .^reg, the angle /3 of rotation of the vector-field VI? along a curve from 2) is uniquely determined by 
the ordered pair of the ends of this curve since VI?(t, r) 7 ^ 0 V(t, r) G 2) by (I4.11|l . Let /3v(®) be the maximum 
of 1/3 1 over all pairs in 2 ). 

Lemma 6.5. Suppose that '3 := A x C C Zreg, where A is an interval and C C is a convex set. Then 


idv(®) < sup |a;v(p)| x diamA -|- sup 

pGS pes 



2 -|- Tp X diamC, 


where diam is the diameter of the set, i.e., the supremum of distances between two its elements. 

Proof: Let pi = {ti,ri) G 2) := A x C, * = 0,1 and to A for the definiteness. In 2), we introduce two 
parametric curves C,{s) := [s,ro],s G [to,tx] and ^(s) := [ti,(l — s)ro -l-sri],s G [0,1], connecting the point 
[to, ro] with [to, ri] and [to, ri] with [ti, ri], respectively. The concatenation of these curves connects pi with p 2 
and lies in 2). So the angle of the gradient rotation when going from pi to p 2 inside 2) is the angle of rotation 
</3(; when going along C(-) plus the angle of rotation <j3^ when going along ^(-j. Let <p[t,r\ stands for the 
orientation angle of VD[t,r] in the absolute coordinate frame. Then 


(vi?aC(s)];r[C(s)]) m 


l|VI?[C(s 


= wv[C(s)] ^ l</3cl = 




wv[C(s)] ds 


'io 


< sup |wv(p)|(ll - to)-, 
pGS) 


d (I?"[C(s)][ri-ro];T[C(s)]) (I?"[C(s)]T[C(s)];T[C(s)]) , y. 

(I?"[C(s)]T[C(s)];iV[C(£)]) m _ r-o;r[C(s)]) - (n - ro;IV[C(s)]) ^ 


||VI?[C(s) 


ds 




< Iki -|</ 35 | < y ds < |[ri - roll sup 


To complete the proof, it remains to take the supremum over po,Pi G 2). 


□ 


Corollary 6.2. Let us consider r] £ (0,7r/2) and the equation D[t,r{s)] = 7 ^ where r{s) := r+sN, N := N{t,r) 
and t,r are given and such that 7 -(t) < 7 = D[t,r] < 7 +(I). Whenever 


I /I / Av . f7-7-(t) 7+(t)-7 


V 




‘'P '^p o\/ 0 -.^ -r b1 

this equation has a solution s such that |s| < I 7 ' — 7 I, where Ay is taken from (14.131) . 


14 

































Proof: We first note that due to (I4.10p . (14.131) . 7 -(t) < D[t,r{s)] < 7 +(t) and |s| < that 

^D[t,r(s)] = (VD[t,r(s)]; A^) = /o[t,T'(s)] {N[t,r{a)\\N) e [cos?7Av;6p]. 


So the premise of this entailment is true whenever 


|s| < min 


1 7-7- ( t ) _ 7 +ft) - 7 ] 

\/K + hi ’ J 



and as s runs over [ — , the value D\t^ 'f'(s)] covers a set that includes the interval [7 — g, 7 + <?]. This 

implies existence of solution, whereas the estimate of |s| follows from the above estimate of ■^D[t, r(s)]. □ 

The distance between the sets A and B is denoted by dist [A]B] := infr^gA.rtgs Ifto — ri,\\. 

Lemma 6 . 6 . Whenever "/-{t) < 71,72 < 7+(0> the following inequalities hold: 

bf^\l 2 - 7i| < dist [/(t, 7 i);/(t, 72 )] < A^ft 72 - 7 i|. (6.14) 

Proof: Since t does not affect the claim, we will notationally ignore it in the proof. Let 71 < 72 for the 
definiteness. Since the isolines I[yi] are compact due to ii) in Remark HTI there exist points ri G = 1,2 

such that dist [ 1 ( 71 ); 1 ( 72 )] = ||r 2 — ri||. Hence for r(s) := ri + s(r 2 — ri), we have 

(7 11:71 . Buol 

—H[r(s)] = (VD[r(s)];r 2 - ri) < ||VD[r(s)]||||r 2 - rill < 6 p||r 2 -ri||, 

/■i d 

72 - 71 = L>[r(l)] - L>[r( 0 )| = J —H[r(s)| ds < fepdist [/( 71 ); ^( 72 )], 

which implies the first inequality in (16.1411 . For any 7 G [ 7 i, 7 _|_), we have by Corollary 16.21 

lim(5“^dist [/(y + (5);/(y)! < Ay^. (6.15) 


Hence 

|dist [/( 7 i );/(7 + (5)| - dist [/( 7 i);/( 7 )| | < dist [/(y + 5);/(y)) -)> 0 as (5 -)> 0 , 
i.e., the map 7 G [ 71 , 7 +) dist [/( 7 i);/( 7 )| is continuous. So for any e > 0, the set 

S' := {7 G [ 71 , 7+1 : dist [/( 7 i);/( 7 )| < (Ay^ +e)(7-7i)} 

is closed and by (16.151) (where 7 = 71 ), contains an interval of the form [ 71 , 7 *!, among which there is a maximal 
one. For it, 7 , = 7 +. Indeed if 7 * < 7 +, (16.1511 (where 7 = 7 *) implies that there exist Jq > 0 such that 
7 * + (5o < 7 +, and for 6 G (0, (5o|, 

dist ^ ^ ^ ( 5 );/( 7 i)| < dist [/(y, + i 5 );/( 7 *)| + dist [/(y*);/(yi)) 

< (Ay^ + e )(5 + (Ay^ + £)(7* — 71) < (Ay^ + £)(7* + (5 — 71) => 7* + <5 G S, 

in violation of the definition of t* as the end of the maximal interval. This contradiction proves that the second 
inequality from (16.141) holds for any 72 G [ 71 , 7 +], with Ay^ replaced by Ay^ + e. The proof is completed by 
letting e —>• 0 . □ 

Lemma 6.7. Suppose that 'y(t) G [ 7 _(t), 7 +(t)| is a smooth function of time t and r* G R^. Then the function 
1 1 —>■ L(t) := dist [/[t, 7 (t)]; r*| is locally Lipschitz continuous and for almost all t, 


Lit) 


< sup |Aft,r)| 

re/[t,7(t)] 


| 7 (t)| sup p ftt,r). 

rG/[t.7(t)] 


(6.16) 


Proof: Given t = t, Corollary 16.21 entails that dist [/[t, 7(t)|;/[t, 7(t)|] —>■ 0 as t —^ t, and the set {r' = 
r + sNfi, r) : r G /[t,7(t)|, |s| < e} is a neighborhood of /[t,7(t)| for small enough e > 0. It follows that /[t,7(t)| 
lies in this neighborhood whenever t Kit, and so any point r' G /[t,7(f)| has the form r' = r + sN(t,r),r G 
/[t, 7(t)|. On the other hand for small enough e, the implicit function theorem guarantees that s = s{r,t) here 
and moreover, the function s(-, •) is smooth. Since 

L(t) = min_ ||r + s(r,t)iV[t,r| — r*|| (6-17) 

rG/[t.7(t)] 
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for t ^ t and the function following the min is continuous in r and locally Lipschitz continuous in t uniformly 
over r G /[?, 7 (f)], the minimum L(t) is also locally Lipschitz continuous [HI Sect. 2.8] and so L(t) exists for 
almost all t. Moreover, whenever L{t) exists and t G S := {t : £)[r, r*] ^ /[t, 7 (r)]}, 


L(t) 


min — jjr + r] - r*|| 

reM dt 




where M is the set of r’s furnishing the minimum in (16.171) . For them 


^ = sgn ( 7 (t) - D\i, r*]), 

whereas 

D[t,r + s{r,t)N{t,r)] = j{t) D[[t,r] + {VD{t,r);N{t,r)) = 7 ( 7 ) ^ ’ 

which implies (16.161) for t = 7 and almost all 7 G S'. Since L(t) = 0 Vt ^ S, the derivative L(t) is zero for almost 
all t ^ S, and so (16.161) does hold as well. □ 

Now we show that the robot arrives at a state where d(t,) >v — 5, (T(t*) < 0 if it starts deep enough in Z^eg. 
Lemma 6.8. Let the robot start at t = to with d < v — 5 and 


7-(to) + A^/2 < D[to,r{to)] < 7+(to) - ^'f/2. 

Then there exists a time t* > to such that 

d{t^) = r — 6, d{t^:) > 0, cr(t*) < 0, 

and _ 

[t,r{t)] G Zreg, ||r(t) - r(to)|| < ^( 2 fc + 1 ) VtG[to,t*]. 

Proof: If d{to) = v — d, d(to) > 0, the claim is clear due to (16.91) . Otherwise either (i(to) < — <5 or 

d(to) = V — 5, d{to) < 0; in both cases, the open set 

S := {t > to : d{t) <v - S,"f-{t) < D[t,r{t)] < 7+(t)} 

contains all t > to that are close enough to to. Let (to,t*) be the leftmost connected component of this set. 

1) Suppose that t* > to+ . By Lemma l6.4l there existsT < , T > 0 such that for t G [to,to+T], 

the robot remains in the disc B of the radius ^(2fc + 1) centered at r{tQ), whereas the vector e{9) rotates 
clockwise through the angle 27r(fc + 1). By (I4.10p . (I4.13L and (I6.16p . the distances from r{to) to I[t,^i{t)] with 
i = ± are reduced by no more than 

(6, + 7Av' )T < (6, + 7Av') • 

By (16.141) . these distances are initially no less than b~^A^l2. By the second inequality from (I4.16|) (with r := S), 
this means that the isolines /[t, 7_(t)] and I[t,^+{t)] do not reach the disc B and so [to, to + T] x B C Z^eg- 
Then (|4.4I) and Lemma [H31 ensure that as t runs from to to to + T, the gradient VD[t,r{t)] turns through an 
angle that does not exceed 

b^T + + bedlam.B < ^ + 2 -^ 6 ^ + bl^{2k + 1 ) < 27Tk, 

fid fio 

where the last inequality follows from the first relation in (14.161) (with v := d). Since meanwhile e(d) turns 
through an angle of 27r(fc + 1), the gradient VD[t, r(T)] and e[9{T)\ are identically directed at some time instant 
T S [to, to + T], Then 

di^) = D[[T,r{T)]+v {VD[T,r{T)];e[9{T)]) = p[t, r(r)]{u- A[r,r(r)]} n, 

in violation of d{t) < v — S Vt G Hence < to + ■ So either 2) Zl[t*,r(t,)] = 7_(t*), or 3) 

r(t,)] = 7+(t*), or 4) 7-(t*) < D[t^, r{t^)] < 7+(t*) and d(t*) = u — 5. 

2,3) By retracing the above arguments, we see that in case 2), the isoline cannot reach the disk B and thus 
the current location of the robot in time t* — to A . Thus case 2) does not occur. Case 3) is similarly 

rejected. 

4) In this case, d[t) < v — 5'it G [to,t*] A d(t*) = ^ — d d(t*) > 0 cr(t*) < 0, where the last implication 
is due to (ESI). The last claim of the lemma follows from Lemma 16.41 □ 

By summarizing the foregoing, now we show that the robot does reach V*(t) with a certain “excess”. 
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Lemma 6.9. In a finite time, the robot reaches the set Vm,{t) := {r : D{t,r) > 7 **(f)}, where 7 **(i) := 
7 *(t) + A.y/2. 

Proof: Suppose to the contrary that the claim is incorrect r{t) ^ D\t,r{t)\ < 7 **(t) Vt > 0, where 

7 **(t) < 7 +(t) — A.y/2 by (14.1311 . We consider separately three cases. 

(a) V — 6 < d(0),CT(0) < 0. This case is impossible due to Corollary 16. II 

(b) V — 5 > (i(0). By Lemma [6.81 along with (14.131) . this case entails (a) at some time instant and so is 
impossible. 

(c) c?(0) > V — 5. Let be the leftmost connected component of the set 

{f > 0 : d{t) > V — 5,^- < D\t, r{t)] < 7 +(t)}. 

Since n — 6 > 0 and by (14.101) . oo > 7 + > 7 +(t) — A — 7/2 > 7 **(t) > D[t, r{t)], we conclude that < 00 . Since 
d{t) >iy — 6>jioitG (0,t*) by (|6.5I) and | 7 -(t)| < 7 , we see that = u — 5. The proof is completed by 
retracing the arguments from b). 

The contradictions obtained complete the proof. □ 

The last lemma in fact completes the proofs of Theorems 14.1114.21 and Remark 14.11 
Lemma 6.10. The robot cannot leave the desired vicinity Vi,{t) of the maximizer when starting at t = t in 

Proof: Suppose to the contrary that the robot leaves 14 (i), starting at t = t with r(T) G 14* (t) ^ 
r(r)] > 7 **(t). By the continuity argument, it intersects first /[47**(t)] and then I[t,^^,{t)] Let > 0 be 
the earliest time t such that d{t) = 7 *(t), and let to be the latest time t G (0,4) such that d{t) = 7 **(t). Then 

d{t) < 7**(t) Vf e {to,ti) ^ d{to) < 7**(to) = 7*(4) <v-5 

by (14.101) and (16.51) . By Lemma IHIH there is > to such that d{t^) = v — 5,a{t,f) < 0, and ||r(t) — r(fo)|| < 
-^{2k + 1) Vt G This inequality and the arguments from 1) in the proof of Lemma 16.81 show that the 

robot does not reach I[t,'-f^{t)] for t G [4,4] and so 7 *(t») < d{t) < 7**(4)- Then for t > t,, the robot reaches 
I[t, 7 +(t)] with constantly maintaining d > v — 6 true, where v — S > 'ji,{t). So d{t), t > to cannot reach the value 
7 *(t) earlier than it reaches 7**(t) for the second time, in violation of the definitions of 4,4- The contradiction 
obtained proves the lemma. □ 

PROOF OF THEOREM [4!^ This theorem is immediate from Lemmas 16.91 and 16.101 □ 

PROOF OF THEOREM [47TJ This theorem is immediate from Theorem 14.21 □ 

PROOF OF REMARK mi} This remark holds since in this appendix. Assumption 14.II was taken in the 
generalized form of i)—iii) from the remark. □ 
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